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Description 

This invention disclosure describes a graphics adapter with frame-buffer compression engine and a network 
interface A network attached graphics adapter is a device similar to a traditional graphics adapter that adds to 
TkZmI^ video output port or ports, such as a DVI port, with a network mterface port or ports 
SEtaet or Infiniband port The network interface port can be used for transrmtting the contents of 
h 7fh£b££* either compressed or uncompressed form or any region of the graphics adapter's memory to 
one o^ore remote network attached devices such as a compute node with a network mterface card This 
capab h^y Sows multiple users to remotely access and view graphics content created by 8^»«dtoM 
SW^es such as high-availability (HA) servers, located anywhere, such as in a backroom, glass 
r^om^ oTan "Internet Data Center (TOC)", using existing network infrastructure and display deuces, such as 
"thin clients" or "appliances". 

Current display refresh on the graphics adapter works as follows: 

While (TRUE) 

If start of vertical retrace 

For all pixels in frame-buffer 

Read pixel from frame-buffer 
Convert pixel to output format 
Transmit out analog/digital port 

The functionality Usted above would be replaced with the following: 

While (TRUE) , , 

If new update is requested a time threshold has been reached 

If there has been a change in some portion of the frame-buffer 
If compression of the frame-buffer enabled 

Compress the frame-buffer , 
Convert the compressed buffer to Internet Protocol (IP) or other network protocol 
Transmit the compressed frame-buffer out the network interface. 



Else 



Convert the frame-buffer to IP or other network protocol 

Transmit the uncompressed frame-buffer out the network mterface port. 



The algorithm listed above could be combined with algorithms presented in the invention disclosure titled 
^S£S^^ redundant data in an Internet Visualization ^Appliance" which mvolved imposing a 
tiling on the image and keeping track of the tiles that have been modified. 

The algorithm listed above could also be modified slightly to handle capabilities such as overlay planes and 

In this scenario, multiple regions of the frame-buffer would be simultaneously 
Z for exSe me primary display surface and overlay plane. From this data the displayed image would be 

5 the frame-buffer compression engine. The following algorithm demonstrates this 



functionality. 



While (TRUE) 

If new update is requested a time threshold has been reached 

If there has been a change in some portion of the primary display surface or overlay planes 
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If compression is enabled 

Coalesce and compress the buffer resulting from the primary and overlay planes 
And cursor location. 

Convert the compressed buffer to Internet Protocol (IP) or other network protocol 
Transmit the compressed frame-buffer out the network interface. 



Else 



Convert the coalesced buffer to IP or other network protocol 
Transmit the uncompressed buffer out the network interface port. 



The operation of coalescing involves reading the pixel from the higher priority buffer or determining the buffer 
that has precedence and performing color LUT conversions and also merging in the cursor. 

One of the disadvantages of current display refresh technology is that the data that is sent out the frame-buffer is 
not tagged to allow delivery of the frame-buffer contents to multiple destinations. Consequently, graphics 
adapters are usually not used in "server" like environments where multiple users share a single machine. As a 
result, technologies like Citrix Metaframe are used that send the graphics protocol over the network which must 
be rendered by the client. 

There have been graphics cards created with multiple video output ports. With these types of video cards it 
might be possible to have one user using one region of the frame-buffer and another user using another region 
of the frame-buffer. It may also be possible to load a computer with more than one graphics adapters and have a 
one-to-one mapping between users and graphics adapters. However, the number of physical ports on the 
graphics adapter and number of graphics cards in the system limits these approaches. 

Using this invention it becomes possible for multiple users/clients to share or time-slice a graphics card and 
have their own unique session and/or desktop. Given a graphics adapter with it's own video memory associate a 
unique region of the graphics adapter memory with each unique network device. For example, if a graphics 
adapter has 32 Mbytes of memory the first 4 Mbytes might be assigned to network device 1, the second 4 
Mbytes may be assigned to network device 2, and so on. The algorithm listed below would then be used to 
update each network-attached device: 

While (TRUE) . u A 

If new update is requested by a specific network device or a time threshold has been reacnea 
If the frame-buffer for the specific network device has been modified 

Configure the frame-buffer compression engine to compress the frame-buffer 

for the network device. 
Compress the frame-buffer for the network device. 
Convert to IP 

Configure the network interface unit to send the compressed frame-buffer to the 
network device. 

Transmit the compressed frame-buffer to the network device out the network 
Interface port. 

Another possible use of this invention is in the area of collaboration where multiple users share a common 
desktop or window within a frame-buffer. In this scenario the same frame-buffer or same region of the frame- 
buffer would be compressed and transmitted to multiple network-attached devices. 
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Advantages over Prior Art 

. Off-loads the compression, read back, and transmittal of the frame-buffer from the host system or CPU. 
Currently to transfer an image rendered in a graphics adapter to a remote network attached device the 
frame-buffer would be required to be read back from the graphics card into system memory, 
compressed, and sent over the host system's network interface. Alternatively, a PCI board or appliance 
could be used as identified in the invention disclosure called "Internet Graphics Appliance . However, 
this invention has additional benefits beyond the IGA approach, which are listed below. 
. Transmittal of the contents of the frame-buffer only occurs when the contents of the frame-buffer 
change Current graphics adapters transmit the entire contents of the frame-buffer to the monitor at a 
continuous refresh rate independent of whether or not any pixels within the frame-buffer have changed 
in value This is necessary because monitors and other display devices do not have their own local 
display refresh memory. As a result, the overhead of display refresh imposes a constant dram on the 
graphics adapters memory sub-system, which significantly impacts graphics rendering performance 
since the same memory is being accessed by the rendering engine and display refresh unit. This 
invention decouples the actual display device refresh from the graphics adapter which could free up 
significant memory bandwidth when actual frame-buffer image updates are occurring less often than the 
video refresh rate of the display device. It is noted that if a network fabric such as Ethernet were used for 
display refresh, the capability presented here would be dependent on the remote network attached 
monitor or display device to be able to persist the transmitted frame-buffer in its own local memoir and 
Io refresh from a local memory source. Currently this technology does not exist and this is the subject of 
another invention disclosure that is in the process of being submitted. 
. Transmittal of the contents of the frame-buffer can be limited to regions of interest. Current video output 
JortsTgTaphics card transmit the entire contents of the frame-buffer at a constant monitor refresh rate 
such as 75 Hz independent of the portions of the frame-buffer that have been modified. Since the 
graphics adapter renders the pixels into the frame-buffer it should be able to easily keep track of the 
extents of the changes or tiles or regions that have changed within the frame-buffer very easily. The 
eraphics adapter needs to only transmit the regions that have changed. 
. Transmittal of frame-buffer contents or some other region of the graphics adapters locd memory could 
beTag^adiessed and sent to more than one network attached device. This wou d allow multiple users 
of a eraphics card to be supported simultaneously with their own session or in a collaborative 
environment. Current video output ports on graphics cards transmit the contents of the frame-buffer out 
the video port to whatever is plugged into it. The contents of the frame-buffer are not tagged and 
transmitted to a destination. This makes it very difficult to support multiple users or multiple display 
devices simultaneously using the same graphics adapter. 
. Lower system cost and higher efficiency can be obtained in comparison to the invention disclosure 
"Internet Graphics Appliance" (IGA) or PCI board solution. The IGA is a separate appliance that grabs 
the incoming frame-buffer at refresh rates and performs many operations on the mcoming pace ^stream 
to identify if the current frame has changed and the regions or tiles that have changed. These changes 
could be more easily and efficiently tracked by a graphics adapter and could be used to identify when 
data in the frame-buffer needs to be transmitted and the regions that are required to be transmitted^ 
Additionally, a separate device with it's own frame-buffer memory and electrical ™™£™™ ts ™ ld ™' 
be required which would further reduce the overall system cost. It is also noted that this capability does 
not currently exist within a graphics adapter. The invention disclosure titled "Internet 
Appliance" presents a methodology that would allow the frame-buffer or region of the frame-buffer to 
be set over the network to a remote display based on technologies and approaches currently available. 
The IGA invention could be viewed as a technology that could be used while transitioning from current 
technologies to the approach identified in this invention. 
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• Graphics adapter frame-buffer can be viewed remotely anytime, anyplace, and anywhere with no 
distance limitations. Current video out technologies has distance limitations and cannot be viewed from 
any network-attached device. 

• Graphics Adapter with Frame-Buffer Compression Engine and Network Interface. 

Possible Embodiments 

This section describes possible embodiments of the invention: 

• The frame-buffer compression engine and networking functionality may be implemented in a separate 
chip or chips on the graphics adapter as shown in Figure 1 Variation 1 . The frame-buffer compression 
engine and networking functionality may be implemented in a custom ASIC chip, DSP, FPGA or other 
alternative. It may be possible to easily design a graphics board with this type of functionality on it 
without modifying the currently available graphics chips. 

• The frame-buffer compression engine and networking functionality may be implemented as a block or 
blocks within the graphics chip as shown in Figure 2 Variation 2. In this design scenario current 
graphics chips would have to be modified significantly to add this type of functionality. 

• The video and DVI port may be eliminated from the graphics adapter completely as shown in Figure 3 
Variation 3. This would reduce the cost of the graphics adapter. 

• The PCI or AGP interface may be eliminated from the graphics adapter in which case the graphics 
commands themselves would be received from the network interface as shown in Figure 4 Variation 4. 
However, the network connection would have to be significantly faster than what is currently available. 

• The PCI or AGP interface could be substituted with another type of network interface such as 
Infinniband or Fiber Channel or another type of network technology that may become available. This 
embodiment is not shown in a diagram, but is easily created by substituting in the diagrams everywhere 
where is states PCI or AGP. 

. Graphics Adapter with Frame-Buffer Compression Engine Only: In this embodiment the graphics 
adapter only has the frame-buffer compression engine and does not have a network interface. In this 
embodiment the graphics card compresses the frame-buffer. The compressed frame-buffer would be sent 
back to the host system and transmitted over the network by the host to the client. 
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